Accelerated Optimization with Orthogonality Constraints

We develop a generalization of Nesterov's accelerated gradient descent method which is designed to deal with orthogonality constraints. To demonstrate the effectiveness our method, we perform numerical experiments that number iterations scales square root condition number, and also compare existing state-of-the-art quasi-Newton methods on Stiefel manifold. Our show outperforms some large, ill-conditioned problems.